State counts for short Markov chains

Imagine we have a multi-step process that can be in any of a finite number of states. At any given time the process will be in one of these states, but at the next time-step it transitions to a new state (which may be the same state though now at a different time). If the probabilities of transitioning to the new state are fixed and independent of the present state, then this process is referred to as a finite-state Markov chain. Given a fixed probability distribution for the initial state where the process starts, as well as a prescribed number n of steps, we can count how many times the process visits each of its possible states. If we statistically add up all of the possible state counts that could arise from all possible processes and then divide by the number of such processes (in effect, “average” them), then we obtain the average proportion of time spent in each state. Alternatively we can describe this histogram as the probabilities of finding the Markov process in each state; that is, the state probability distribution associated with this Markov chain. As n becomes large the problem approaches a steady state that is predicted quite reliably and already researched by various sources. But if n is small then these predictions are not very accurate. This report describes a computational method for finding these distributions both exactly and quickly. [Although it is not necessary to go into a thorough explanation of the con-